Fuzzy modelling via on-line support vector machines

Fuzzy systems can approximate any continuous nonlinear function to arbitrary accuracy, provided that suitable fuzzy rules are available (Wang 1994). Recent results show that the fusion of some intelligent technologies with fuzzy systems seems to be very effective for nonlinear systems modelling. In Oh, Pedrycz, and Roh (2006), fuzzy neural networks endowed with polynomial neurons were investigated, which also involved mechanism of genetic algorithms. The feature wrapper and feature filter approaches were proposed in Uncu and Turksen (2007) to realise fuzzy modelling. Straszecka (2006) gave an unified fuzzy-probabilistic framework for modelling processes. The key problem of the fuzzy modelling is extraction of the fuzzy rules, it can be divided into two classes (Leski 2005): (1) obtaining fuzzy rules from experts, (2) obtaining fuzzy rules automatically from observed data. The expert method uses the unbias criterion (Rivals and Personnaz 2003) and the trialand-error technique, it can only be applied off-line. The process of fuzzy rule extraction for nonlinear systems modelling is called structure identification. A common method is to partition the input and the output data, it is also called fuzzy grid (Jang 1993). Most of the structure identification approaches are based on off-line data clustering, such as fuzzy C-means clustering (Mitra and Hayashi 2000), mountain clustering (Mitra and Hayashi 2000) and subtractive clustering (Chiu 1994). These approaches require that the data is ready before the modelling. Besides clustering approaches, fuzzy rule extraction can also be realised by the neural networks method (Jang 1993), genetic algorithms (Rivals and Personnaz 2003), SVD-QR (Chiang and Hao 2004) and the support vector machine (SVM) technique (Cristianini and Shawe-Taylor 2000). SVM was first used for solving the pattern classification problem. Vapnik defined it as structure risk minimisation, which minimises the upper bound of the modelling error. The basic idea of SVM modelling is to map the inputs into a higher dimensional feature space, then solve quadratic programming (QP) with an appropriate cost function (Mueller, Mika, Rasch, Tsuda, and Scholkopf 2001). There is one important property in SVM: the solution vector is sparse. Only the non-zero solutions, which are called support vectors, are useful for the model. Almost all nonlinear system identifications via SVM are off-line, because they are batch processes (Engel, Mannor, and Meir 2004). There is a big obstacle for using on-line SVM, the dimension of the kernel will increase as the time passes by. To the best of our knowledge, online identification via SVM has still not been applied in the literature. In this article, we use on-line support vectors to extract the fuzzy rules. For the parameter identification, we use the data to modify the membership functions of each fuzzy rule. Stability of learning algorithms for the parameter identification are very important in applications. It is well known that normal identification algorithms (e.g. gradient descent and least square) are stable in ideal conditions. They might become unstable with respect to unmodelled dynamics, some robust modification techniques are needed (Ioannou and Sun 1996). By using passivity theory, we successfully proved that neural networks with time-varying learning rates are stable and robust to any bounded uncertainties (Yu and Li 2001). Does the parameter identification of the fuzzy modelling have the similar characteristics?